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(57) Abstract 

A rearview vision system for a vehicle includes at least one image capture device directed rearwardly with respect to the direction 
of travel of the vehicle. A display system displays an image synthesized from output of the image capture device. The display system 
is preferably contiguous with the forward field of view of the. vehicle driver at a focal length that is forward of the vehicle passenger 
compartment. A plurality of image capture devices may be provided and the display system displays a unitary image synthesized from 
outputs of the image capture devices which approximates a rearward-facing view from a single location, such as forward of the vehicle. 
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KF ARV1EW VISION SYSTEM FOR VEHICLE INCLUDING PANORAMIC VIEW 
CROSS-REFERENCE TO RELATED APPLICATION 
This application is a continuation-in-part of patent application Serial No. 
08/445,527 filed May 22, 1995, which is a continualion-in-part of patent application Serial 
5 No. 08/023,918 filed February 26, 1993. 

BACKGROUN D OF THE INVENTION 
This invention relates generally to vision systems for vehicles and, more 
particularly, to rearview vision systems which provide the vehicle operator with scenic 
information in the direction rearward of the vehicle. More particularly, the invention 
10 relates to a rearview vision system utilizing image capture devices, such as CMOS imaging 
arrays and the like. 

A long-felt need in the art of vehicle rearview vision systems is to reduce the 
amount of time spent gathering information of the condition around the vehicle in order to 
safely carry out a vehicle maneuver, such as a turn or a lane change. It has been 
15 determined that up to about 50 percent of maneuver time is spent gathering information 
with conventional rearview mirrors. This typically requires viewing one or more mirrors, 
turning the head and body to check blind spots, and double-checking the mirrors prior to 
executing the maneuver. Some improvement has been obtained by adjusting mirror optics, 
for example, to increase the observed field of view rearward of the vehicle. However, this 
20 is achieved with an increase in image distortion which makes driver awareness of 
conditions to the side and rear of the vehicle even more difficult. 

Another long-felt need in the art of vehicle rearview vision systems has been to 
eliminate exterior rearview mirrors by utilizing image capture devices, such as cameras, in 
combination with dashboard displays. This would be beneficial because it would reduce 
25 wind drag on the vehicle, wind noise and vehicle weight. Furthermore, rearview mirrors 
protrude a substantial distance from the side of the vehicle, which makes maneuvering in 
tight spaces more difficult. Image capture devices are capable of positioning in a greater 
variety of locations on the vehicle, providing more flexibility of vehicle styling. It is 
further expected that camera systems would greatly reduce the blind spots to the sides and 
30 rear of the vehicle common with vehicles equipped with conventional rearview mirror 
systems. The driver cannot perceive vehicles, objects, or other road users in such blind 
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spots without turning his or her body, which interferes with forward-looking visual 
activities. 

Camera-based rearview vision systems for vehicles have not obtained commercial 
acceptance. One difficulty with proposed systems has been that they present a large 

5 amount of visual information in a manner which is difficult to comprehend. This difficulty 
arises from many factors. In order to significantly reduce blind spots, multiple image 
capture devices are typically positioned at various locations on the vehicle. The image of 
an object behind the equipped vehicle is usually captured by more than one image capture 
device at a time and displayed in multiple images. This may confuse the driver as to 

10 whether more than one object is present. When multiple image capture devices are 

positioned at different longitudinal locations on the vehicle, objects behind the vehicle are 
at different distances from the image capture devices. This results in different image sizes 
for the same object. This effect is especially noticeable for laterally extending images, 
such as a bridge, highway crosswalk markings, the earth's horizon, and the like. Such 

15 images are at different vertical angles with respect to the image capture devices. This 
results in different vertical positions on the display causing the elongated image to appear 
disjointed. 

A camera system provides a monocular view of the scene, compared to the 
binocular stereoscopic view obtained when the scene is viewed through a rearview mirror. 
20 This makes the ability to judge distances in a camera system a problem. This effect is most 
noticeable at distances close to the vehicle where stereoscopic imaging is relied upon 
extensively by the driver in judging relative locations of objects. Therefore, known camera 
systems fail to provide to the driver important information where that information is most 
needed at small separation distances from surrounding objects. 

25 Another difficulty with camera systems is that, in order to provide a sufficient 

amount of information, the camera system typically presents the driver with a greatly 
increased field of view. This improves performance by further reducing blind spots at the 
side and rear of the vehicle. However, an increased field of view is often obtained by 
utilizing a wide-angle lens which introduces distortion of the scene and further impairs the 

30 ability of the driver to judge distances of objects displayed. The problem with such 

distortion of the scene is that the driver must concentrate more on the display and take a 
longer time to interpret and extract the necessary information. This further distracts the 
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driver from the primary visual task of maintaining awareness of vehicles and other objects 
in the vicinity of the driven vehicle. 

Yet an additional difficulty with camera systems is that fiai panel displays present 
the image captured by the rearward-facing image capture device, or devices, at a focal 
5 length that approximates the arm length of the vehicle driver. In order to observe the 
condition of the vehicle utilizing the rearview vision system, the driver must change bis or 
her gaze from the forward field of view of the vehicle to the display. Because the forward 
field of view of the vehicle is at a focal length that is much greater than the focal length of 
the displayed image, the eyes of the driver must refocus upon changing gaze. This 
10 refocusing further increases the amount of time for the driver to assimilate the information 
in the displayed image. Furthermore, when the gaze of the driver returns to the forward 
field of view, the eyes must, again, refocus to the greatly longer distance. 

Yet an additional difficulty with camera systems is that of finding adequate space in 
the crowded area of the vehicle's dashboard for the components making up the display. 
15 SUMMARY OF THE INVENTION 

The present invention is directed towards enhancing the interpretation of visual 
information in a rearview vision system by presenting information in a manner which does 
not require significant concentration of the driver or present distractions to the driver. This 
is accomplished according to an aspect of the invention in a rearview vision system having 
20 at least two image capture devices positioned on the vehicle and directed generally 

rearwardly with respect to the direction of travel of the vehicle. A display is provided for 
images captured by the image capture devices. The display displays an image synthesized 
from outputs of the image capture devices which approximates a rearward-facing view 
from a single location. In order to obtain all of the necessary information of activity, not 
25 only behind but also along side of the vehicle, the virtual camera should be positioned 

forward of the driver. The image synthesized from the multiple image capture devices may 
have a dead space which corresponds with the area occupied by the vehicle. This dead 
space is useable by the driver's sense of perspective in judging the location of vehicles 
behind and along side of the equipped vehicle. 
30 The present invention provides techniques for synthesizing images captured by 

individual, spatially separated, image capture devices into such ideal image, displayed on 
the display device. This may be accomplished, according to an aspect of the invention, by 
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providing at least three image capture devices. At least two of the image capture devices 
are side image capture devices mounted on opposite sides of the vehicle. At least one of 
the image capture devices is a center image capture device mounted laterally between the 
side image capture devices. A display system displays an image synthesized from outputs 
5 of the image capture devices. The displayed image includes an image portion from each of 
the image capture devices. The image portion from the center image capture device is 
vertically compressed. 

It has been discovered that such vertical compression substantially eliminates 
distortion resulting from the spatial separation between the cameras and can be readily 

10 accomplished. In an illustrated embodiment, the image compression is carried out by 

removing selective ones of the scan lines making up the image portion. A greater number 
of lines are removed further away from the vertical center of the image. 

The compression of the central image portion produces a dead space in the 
displayed image which may be made to correspond with the area that would be occupied by 

15 the vehicle in the view from the single virtual camera. Preferably, perspective lines are 
included at lateral edges of the dead space which are aligned with the direction of travel of 
the vehicle and, therefore, appear in parallel with lane markings. This provides visual 
clues to the driver's sense of perspective in order to assist in judging distances of objects 
around the vehicle. 

20 According to another aspect of the invention, image enhancement means are 

provided for enhancing the displayed image. Such means may be in the form of graphic 
overlays superimposed on the displayed image. Such graphic overlap may include indicia 
of the anticipated path of travel of the vehicle which is useful in assisting the driver in 
guiding the vehicle in reverse directions. Such graphic overlay may include a distance grid 

25 indicating distances behind the vehicle of objects juxtaposed with the grid. 

According to yet an additional aspect of the invention, a rearview vision system for 
a vehicle includes at least one image capture device positioned on the vehicle and directed 
generally rearwardly with respect to the direction of travel of the vehicle. A display 
system is provided which displays a rear image synthesized from an output of the image 

30 capture device. The rear image is substantially contiguous with the forward field of view 
of the vehicle driver and at a focal length that is forward of the vehicle passenger 
compartment and preferably within the depth of field of a vehicle driver viewing a distant 
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object. Because the image has a focal length that more closely matches that of the forward 
field of view observed by the driver, the need for the driver's eyes to refocus from the 
forward field of view to a much shorter focus distance each time the gaze of the driver is 
directed at the display system is minimized. This reduces the amount of time required for 
5 the driver to gaze at the displayed image and interpret objects displayed in the image. 
Furthermore, the reduction in the repeated refocusing of the driver's eyes reduces driver 
fatigue. If there are any near field objects in the periphery of the driver's forward field of 
view, such as windshield wipers, windshield frame, dashboard, and the like, the display 
system is preferably positioned in a manner which blocks the view of such near field 
10 objects. In this manner, the driver's gaze may shift between the forward field of view and 
the long focal length display system without being refocused on the near field objects. This 
is based upon a recognition that the driver's eyes will tend to refocus on the near field 
object momentarily even though the gaze is being redirected between the forward field of 
view and the display system. 
15 According to yet an additional aspect of the invention, a rearview vision system for 

a vehicle includes at least one image capture device positioned on the vehicle and directed 
generally rearwardly with respect to the direction of travel of the vehicle. A display 
system is provided for displaying a rear image captured by the image capture device. The 
displayed image is a unitary image having an aspect ratio that is between approximately 4:1 
20 and approximately 2:1. In a most preferred embodiment, the image has ail aspect ratio that 
is approximately 8:3. The aspect ratio, according to this aspect of the invention, is 
especially useful where the unitary image is synthesized from a plurality of images which 
are captured by a plurality of image captured devices and are tiled by the display device. 

According to yet an additional aspect of the invention, a rearview vision system for 
25 a vehicle includes a plurality of image capture devices positioned on the vehicle and 
directed generally rearwardly with respect to the direction of travel of the vehicle. A 
display system which includes at least one image generator and an optical correction system 
is provided which displays an image synthesized from outputs of the image capture devices 
as a unitary image. Alternatively, the display system may include a plurality of image 
30 generators, each associated with one or more of the image capture devices and an optical 
correction system which amplifies images generated by the image generators and merges 
them into a unitary image. The optical correction system additionally increases the focal 
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length, or lengths, of the image, or images, generated by the image generator, or 
generators. The display system may be an opaque projection display which is positioned 
approximately at the driver's arm length in front of the driver. Alternatively, the display 
system may be a view-through heads-up display which projects the unitary image onto a 
5 combiner in order to combine the unitary image with the forward field of view of the 
driver. 

These and other objects, advantages, and features of this invention will become 
apparent by review of the following specification in conjunction with the drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 Fig. 1 is a top plan view of a vehicle having a rearview vision system according to 

the invention; 

Fig. 2 is a side elevation of the vehicle in Fig. 1; 
Fig. 3 is a front elevation of a display according to the invention; 
Fig. 4 is the same view as Fig. 1 illustrating an alternative embodiment of the 
15 invention; 

Fig. 5 is a block diagram of an electronic system according to the invention; 
Fig. 6 is the same view as Fig. 3 illustrating an alternate mode of operation of the 
system; 

Fig. 7 is the same view as Fig. 2 illustrating an alternative embodiment of the 
20 invention; 

Fig. 8 is the same view as Fig. 3 illustrating an alternative embodiment of the 
invention; 

Fig. 9 is the same view as Figs. 1 and 4 illustrating an alternative embodiment of 
the invention; 

25 Fig. 10 is the same view as Figs. 3 and 8 illustrating an alternative embodiment of 

the invention; 

Fig. 11 is a chart illustrating the horizontal row of pixels (nl, n2) on which an 
object will be imaged from two longitudinally separated image capture devices as that 
object is spaced at different longitudinal distances from the image capture devices; 
30 Fig. 12 is a forward elevation of a vehicle passenger compartment as viewed by a 

vehicle driver; 

Fig. 13 is a sectional view taken along the lines XIII-XIII in Fig. 12; 
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Fig. 14 is a sectional view taken along the lines XIV-XIV in Fig. 12; 
Fig. 15 is the same view as Fig. 14 of an alternative embodiment; 
Fig. 16 is the same view as Fig. 14 of another alternative embodiment; 
Fig. i / is an enlarged view of the display system in Fig. 14 illustrating details 
5 thereof; 

Fig. 18 is a block diagram similar to Fig. 5 of an alternative embodiment of the 
invention; 

Fig. 19 is a side elevation similar to Fig. 2 of an alternative embodiment of the 
invention; 

10 Fig. 20 is an enlarged side elevation of an image capture device with portions of the 

housing removed in order to reveal internal structure thereof; 

Fig. 21 is a block diagram similar to Fig. 5 of another alternative embodiment of 
the invention; 

Fig. 22 is a block diagram similar to Fig. 5 of another alternative embodiment of 
15 the invention; 

Fig. 23 is a block diagram similar to Fig. 5 of another alternative embodiment of 
the invention; 

Fig. 24 is a block diagram similar to Fig. 5 of another alternative embodiment of 
the invention; and 

20 Fig. 25 is a block diagram of a rearview vision system having extended dynamic 

range capabilities. 

DESCRIPTION OF THE PREFERRED E MBOD IM ENT 
Referring now specifically to the drawings, and the illustrative embodiments 
depicted therein, a vehicle 10, which may be an automobile, a light truck, a sport utility 

25 vehicle, a van, a bus, a large truck, or the like includes a rearview vision system, generally 
illustrated at 12, for providing a driver of the vehicle with a view rearwardly of the vehicle 
with respect to the direction of travel T of the vehicle (Fig. 1). Vision system 12 includes 
at least two side image capture devices 14 positioned, respectively, on opposite sides of 
vehicle 10 and a center image capture device 16 positioned on the lateral centerline of the 

30 vehicle. All of the image capture devices are directed generally rearwardly of the vehicle. 
Rearview vision system 12 additionally includes an image processor 18 for receiving data 
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signals from image capture devices 14, 16 and synthesizing, from the data signals, a 
composite image 42 which is displayed on a display 20. 

As will be set forth in more detail below, the images captured by image capture 
devices 14, 16 are juxtaposed on display 20 by image processor 18 in a manner which 
5 approximates the view from a single virtual image capture device positioned forwardly of 
the vehicle at a location C and facing rearwardly of the vehicle, with the vehicle being 
transparent to the view of the virtual image capture device. Vision system 12 provides a 
substantially seamless panoramic view rearwardly of the vehicle without duplicate or 
redundant images of objects. Furthermore, elongated, laterally-extending, objects, such as 
10 the earth's horizon, appear uniform and straight across the entire displayed image. The 

displayed image provides a sense of perspective, which enhances the ability of the driver to 
judge location and speed of adjacent trailing vehicles. 

Each of side image capture devices 14 has a field of view 22 and is aimed 
rearwardly with respect to the vehicle about an axis 24 which is at an angle, with respect to 
15 the vehicle, that is half of the horizontal field of view of the image capture device. In this 
manner, each of the image capture devices 14 covers an area bounded by the side of the 
vehicle and extending outwardly at an angle defined by the horizontal field of view of the 
respective side image capture device. Center image capture device 16 has a horizontal field 
of view 26, which is symmetrical about the longitudinal axis of the vehicle. The field of 
20 view of each side image capture device 14 intersect the field of view of center image 
capture device 16 at a point P which is located a distance Q behind vehicle 10. 

Rear blind zones 30 are located symmetrically behind vehicle 10 extending from the 
rear of the vehicle to point P. Side blind zones 25 located laterally on respective sides of 
the vehicle extend rearwardly of the forward field of view 36 of the driver to the field of 
25 view 22 of the respective side image capture device 14. An object will not be captured by 
side image capture devices 14 or center image capture device 16 if the object is entirely 
within one of the blind zones 25, 30. In order for an object, such as another vehicle V or 
other road user travelling to the side of vehicle 10, to be observed by an operator of 
vehicle 10, the object must be either at least partially within the forward field of view 36 of 
30 the driver or be captured by image capture devices 14, 16 and displayed on display 20. 

Fig. 4 illustrates vehicle 10 travelling on a three-lane highway having lanes LI, L2, and L3 
with the vehicle in lane L2. Another vehicle V is shown positioned mostly within one of 
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the blind zones 25, but with the rearmost portion of the vehicle V extending into field of 
view 22 where the vehicle image will be captured by one of side image capture devices 14. 
In the illustrated embodiment, vehicle V is a motorcycle travelling in the center of lanes LI 
or L3 and represents a worst case for observing a vehicle travelling at least partially within 
5 one of the blind zones 25. In order for a portion of vehicle V to be extending either 
forwardly or rearwardly of the respective blind zone 25, where the vehicle V may be 
observed by either the forward field of view 36 of the driver or by the rearview vision 
system 12, the field of view 22 of side image capture devices 14 must be sufficiently wide 
to capture a portion of vehicle V as illustrated in Fig. 4. Preferably, the horizontal field of 

10 view 22 of side image capture devices 14 is no greater than that required to provide 

sufficient coverage which would be in the range of between approximately 55 degrees and 
approximately 70 degrees. In the illustrated embodiment, the horizontal field of view 22 is 
61 degrees. In order for a portion of vehicle V to be within a vertical field of view 40 of 
one of side image capture devices 14, the field of view should extend to the pavement at a 

15 plane M which intersects vehicle V (Fig. 2). Preferably, vertical field of view 40 is 
between approximately 60 degrees and approximately 75 degrees. In the illustrated 
embodiment, vertical field of view 40 is 66 degrees. 

A left overlap zone 32 and a right overlap zone 34 extend rearward from respective 
points P where the horizontal fields of view of the side image capture devices intersect the 

20 field of view of center image capture device 16. Overlap zones 32, 34 define areas within 
which an object will be captured both by center image capture device 16 and one of the 
side image capture devices 14. An object in an overlap zone 32, 34 will appear on display 
20 in multiple image portions in a redundant or duplicative fashion. In order to avoid the 
presentation of redundant information to the driver, and thereby avoid confusion and 

25 simplify the task of extracting information from the multiple images or combined images 
on display 20, the object should avoid overlapping zones 32, 34. In practice, this may be 
accomplished to a satisfactory extent by moving points P away from the vehicle and 
thereby increasing distance Q. It is desirable to increase distance Q to a length that will 
exclude vehicles travelling at a typical separation distance behind vehicle 10 from 

30 overlapping zones 32, 34. This separation distance is usually a function of the speed at 

which the vehicles on the highway are travelling. Therefore, optionally, distance Q may be 
made variable, not fixed. In such embodiment, the faster the vehicles are travelling, the 
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further Q should be moved behind vehicle 10 to keep overlap zones 32 and 34 outside of 
the recommended vehicle spacing. If, however, the vehicles are travelling at a slower 
speed, then the generally accepted recommendation for vehicle spacing decreases and it is 
more likely that a vehicle will be within overlap zone 32, 34. Therefore, the distance Q 
5 may be selected to accommodate expected vehicle spacing for an average driving speed of 
vehicle 10. 

Distance Q is a function of the effective horizontal field of view 26 of center image 
capture device 16. As field of view 26 decreases, points P move further rearward of the 
vehicle from a distance Q I? to a distance Q 2 , as best illustrated in Fig. 4. In order to 
10 increase distance Q to eliminate redundant and duplicative information displayed on display 
20 for most driving conditions of vehicle 10, field of view 26 is preferably less than 12 
degrees. In the illustrated embodiment, field of view 26 is between 6 and 8 degrees. 
Alternatively, distance Q may be dynamically adjusted according to some parameter, such 
as the speed of vehicle 10. This would allow Q to be greater when the vehicle is travelling 
15 at a faster speed, where vehicle separation tends to be larger, and vice versa. Field of view 
26 may be adjusted by utilizing a selective presentation of pixels of the captured image in 
the displayed image. 

Referring to Fig. 3, image display device 20 displays a composite image 42 made 
up of a left image portion 44, a right image portion 46, and a center image portion 48. 
20 Each image portion 44-48 is reversed from the image as captured by the respective image 
capture device 14,16 utilizing conventional techniques. These techniques include reading 
the image in reverse with the image capture device, writing the image in reverse to display 
device 20, or reversing the image in image processor 18. Left image portion 44 is joined 
with central image portion 48 at a boundary 50. Central image portion 48 is joined with 
25 right image portion 46 at a boundary 52. As may best be seen in Fig. 3, the image 

portions at boundaries 50 and 52 are continuous whereby composite image 42 is a seamless 
panoramic view rearwardly of the vehicle. As also is apparent from Fig. 3, central image 
portion 48 is narrower than either left image portion 44 or right image portion 46. This is 
a result of reducing the horizontal field of view 26 of center image capture device 16 
30 sufficiently to move points P, and thus overlap zones 32 and 34, a sufficient distance 

behind vehicle 10 to reduce redundant and duplicative images between image portions 44- 
48. Composite image 42 provides a clear image, which avoids confusion and simplifies the 
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task of extracting information from the multiple image portions 44-48. As also may be 
seen by reference to Fig. 3, display 20 may additionally include indicia such as the readout 
of a compass 54, vehicle speed 56, turn signals 58, and the like as weii as other graphical 
or video displays, such as a navigation display, a map display, and a forward-facing vision 
5 system. In this manner, rearview vision system 12 may be a compass vision system or an 
information vision system. 

In the embodiment of rearview vision system 12 having a dynamically adjusted 
value of distance Q, the spacing between boundaries 50 and 52 will dynamically adjust in 
sequence with the adjustment of distance Q. Thus, as overlap zones 32, 34 move further 
10 away from the vehicle; for example, in response to an increase in speed of the vehicle, 
boundary lines 50 and 52 will move closer together and vice versa. In this manner, 
composite image 42 is dynamic, having image portions of dynamically adaptive sizes. 

Display 20 is of a size to be as natural as possible to the driver. This is a function 
of the size of the displayed image and the distance between the display and the driver. 
15 Preferably, the displayed image simulates an image reflected by a rearview mirror. As 
such, the size of the displayed image is approximately the combined areas of the three 
rearview mirrors (one interior mirror and two exterior mirrors) conventionally used with 
vehicles. As best seen by reference to Figs. 2 and 12, display 20 is preferably positioned 
within the driver's physiological field of view forward of the vehicle, generally illustrated 
20 at 70, through the windshield 72 without significantly obstructing the forward field of 
view. It is known that the driver's field of view, with the head and eyes fixed forward, 
extends further in a downward direction than in an upward direction. Display 20 could be 
located above the field of view 70 wherein the display may be observed at the upward 
portion of the drivers field of view. However, the position for the display illustrated in 
25 Figs. 2 and 12 is preferred wherein the display is within the lower portion of the driver's 
field of view. 

Display 20 may be a direct view flat panel display, such as a back-lit or reflective 
liquid crystal display, a plasma display, a field emission display, a cathode ray tube 
electroluminescent, light-emitting diode or deformable mirror display. The display may be 
30 mounted/attached to the dashboard, facia or header, or to the windshield at a position 

conventionally occupied by an interior rearview mirror. However, the synthesized image 
could be displayed using other display techniques such as to provide a projected or virtual 
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image. Alternatively, a virtual image may be displayed on an opaque display adjacent the 
forward field of view. Alternatively, a virtual image may be displayed on a view-through 
heads-up display in which the image is superimposed on the forward field of view. 

In the embodiment illustrated in Figs. 12-17, display 20 displays an image at a focal 
5 length that is forward of the passenger compartment of vehicle 10. Preferably, the image 
displayed by display 20 is at a focal length that is within the driver's normal depth of field 
when viewing a distant object. Display 20 includes an image generator 74, which produces 
an image captured by one or more image capture devices 14, 16, and an optical correction 
system 76, which increases the focal distance of the image generated by image generator 

10 74. In the illustrated embodiment, optic correction system 76 increases the focal distance 
by collimating the rays, generally indicated at X, from diverging rays to generally parallel 
rays projected from the display. Optical correction system 76 additionally magnifies the 
image. In the illustrated embodiment, the magnification is a factor of two. In this manner, 
optical corrective system 76 has the advantage of extending the focal distance of the image 

15 generated by image generator 74 and enlarging the image by the factor of magnification 
thereof. This advantageously allows each image generator 74 to project all or a portion of 
an image captured by one of image capture devices 14, 16, or a combination of portions of 
images from one or more image capture devices, by tiling the images or image portions. 
This is accomplished because the images projected from optical correction system 76 may 

20 abut even though the corresponding 

image generators 74 do not. This provides a convenient technique for joining the images 
synthesized from the image capture devices into a unitary image which represents a 
panoramic view rearward of the vehicle. 

In the embodiment illustrated in Fig. 14, display 20 is an opaque projection display 

25 which projects the image directly toward the driver. In the embodiment illustrated in Fig. 
15, a display device 120 is a view-through heads-up display in which the rays X are 
projected generally upwardly by image generator 74 and optical correction system 76 
which are generally vertically aligned, or aligned forward of vertical. The rays X are 
reflected off a first surface of windshield 72 in the direction of the driver. Windshield 72 

30 acts as a combiner which combines the image displayed by display 120 with a portion of 
the forward field of view 70 observed by the driver. In the embodiment illustrated in Fig. 
15, a combiner other than the windshield may be used. Examples may be holographic or 
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diffractive optical film elements or beam splitters of metal or dielectric thin films. 
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rectangle, around rear image 42. This provides a border around the image to differentiate 
the rear view from the view forward of the vehicle. 
5 In the embodiment illustrated in Fig. 16, display 20A is oriented at an angle with 

respect to the forward field of view of driver D. Image rays X are reflected by a mirror 
140 toward the driver. Display 20A is an opaque display, with mirror 140 blocking the 
driver's view of near field objects, such as wipers 98 and the like. Display 20A has the 
advantage of being capable of location within a forward portion of the dashboard. 

10 Additionally, the only portion of the display visible to the driver is mirror 140. This 
allows near field portions of the display to be significantly reduced. 

Because display 20 has a relatively long focus distance, display 20 defines an 
observation cone, generally designated 78, within which the displayed image can be 
observed. Therefore, the head of the driver must be properly oriented with respect to 

15 observation cone 78 in order to allow the driver to observe the displayed image. However, 
drivers come in various sizes. Therefore, a driver may be too tall or too short to have his 
or her head properly positioned within observation cone 78. In order to provide for 
various size drivers along with various adjustments in seating positions and the like, an 
accommodation means, generally illustrated at 80, may be provided in order to 

20 accommodate variations in the relationship between a driver's head and observation cone 
78. In the illustrated embodiment, accomodation means 80 includes adjustment means 82 
for adjusting the position of observation cone 78. The adjustment means may adjust the 
position of the observation cone either vertically, horizontally, or both vertically and 
horizontally. A vertical adjustment means 82 is illustrated in Fig. 13 in which the 

25 adjustment means includes an electric actuator 84 which is joined by linkage 86 with a 

portion of a housing 88 of display 20. Actuator 84 is electrically interconnected through a 
reversing switch 90 with a driver-operated actuator 92 which may be positioned on 
dashboard 94 or other convenient position accessible to the driver. Housing 88 may be 
adjustably mounted, such as by a pivot 96, in order to allow housing 88 to be adjustably 

30 repositioned with respect to dashboard 94. In this manner, by operation of actuator 92, 
housing 88 may be pivoted upwardly or downwardly with respect to pivot 96 in order to 
adjust the direction of observation cone 78. In this manner, the location of observation 
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cone 78 may be adjusted in order to coincide with the location of the driver's head. In a 
similar fashion, the position of observation cone 78 may be adjusted laterally, if desired. If 
a view-through heads-up display of the type illustrated in Fig. 15 is utilized, the position of 
tne observation cone may be adjusted vertically and laterally, in a similar manner, by 
5 mechanical or optical adjustments of display 20. 

Accomodation means 80 may include extending the rearward field of view displayed 
by display 20 laterally outwardly with respect to the bezel 89 beyond that normally 
observed by a driver. In this manner,. a driver's head located generally centrally within 
observation cone 78 will observe a view generally rearwardly of the vehicle. As the 

10 driver's head is moved laterally within observation cone 78, the driver will observe images 
more laterally to the side of the vehicle as would occur if the driver's head were to be 
moved with respect to a conventional optical rearview mirror system. 

Vehicle 10 may include one or more near field view objects adjacent forward field 
of view 70. One such object is a windshield wiper 98 of the vehicle. Other such objects 

15 may include the top of dashboard 94, the frame around windshield 72, the hoodline, and 
the like. The housing of display 20 in Fig. 14 and mirror 140 in Fig. 15 are positioned 
with respect to forward field of view 70 such that housing 88 or mirror 140 covers any 
near field of view objects in the portion of the forward field of view adjacent display 20, 
20A. In this manner, the gaze of the driver can switch between forward field of view 70 

20 and the image displayed on display 20, without the eyes of the driver focusing on any 

significant near field objects. This is based upon a discovery that, even though the eyes of 
the driver are switching between the long focal distance of the forward field of view and 
the long focal distance of the image displayed by display 20, the eyes of the operator will 
unconsciously momentarily focus on any near field object positioned between the long focal 

25 distance views. Therefore, by blocking the driver's gaze of any near field objects, the eyes 
of the driver will be less stimulated to refocus during the transition from field of view 70 to 
display 20 and back again. 

Image processor 18, which supplies a video signal 100 to image generator 74, may 
have a second input 102 which modulates the intensity level of the image generated by 

30 image generator 74 and displayed by display 20 (Fig. 14). The illumination level of the 
display is set in response to an ambient light input 104 which is an indication of the 
ambient light level around vehicle 10. Image processor 18 responds to the value of 
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ambient light input 104 by producing a luminance intensity signal 102 which increases the 
intensity of the display in response to increases in ambient light level and decreases the 
intensity of the display in response to decreases in ambient light level. However, the level 
of display luminance may be limited to vary between upper and lower limits such that, 
5 once ambient light reaches a particular upper level, further increases in ambient light level 
will not result in a further increase in display intensity. Likewise, once the ambient light 
level decreases below a particular value, further reductions in ambient light level will not 
result in further reduction in display intensity. Ambient light input 104 may be produced 
by a separate ambient light sensor of the type which produces a continuously variable 
10 output in response to variations in ambient light levels, in which case, the intensity of 

display 20 may be proportionately adjusted. Alternatively, ambient light input 104 may be 
produced by a vehicle headlight control system (not shown) which switches the vehicle 
headlights on, or to a nighttime condition, in response to decreases in ambient light levels 
and switches the vehicle headlights off, or to a daytime running light condition, in response 
15 to increasing ambient light levels. Such system is disclosed in commonly assigned United 
States patent application Serial No, 08/277,674 filed on July 19, 1994, by Kenneth L. 
Schierbeek and Niall R. Lynam for an AUTOMATIC REARVIEW MIRROR SYSTEM 
WITH AUTOMATIC HEADLIGHT ACTIVATION, the disclosure of which is hereby 
incorporated herein by reference. If the ambient light signal supplied to ambient light input 
20 104 is a binary signal representative of a daytime ambient light level and a nighttime 
ambient light level, image processor 18 would typically provide a signal on luminance 
intensity line 102 that would switch the intensity level of display 20 between two intensity 
levels. Alternatively, ambient light input 104 may be supplied with a signal developed by 
one or more image capture devices 14, 16. The ambient light signal would be based upon 
25 an average intensity value sensed by all, or a group of, pixels in the image capture device 
or devices. This embodiment eliminates the necessity for a separate ambient light sensor. 
Alternatively, ambient light input 104 may be responsive to manual actuation of the 
vehicle's headlights by the driver. Additionally, a comfort level setting may be provided to 
allow the driver to adjust to a preferred brightness at one ambient light condition. 
30 Thereafter, the system automatically adjusts display brightness according to ambient light 
changes. 
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In the illustrated embodiment, display 20 incorporates a combined image generator 
and optical correction system 106 which provides for both image magnification and light 
ray coiiirnation. In this manner, the image projected from display 20 is larger than the 
image generated by image generator 74 and has a focal length that is greater than the 
5 separation distance between the image generator and the driver and, preferably, is 

generally at infinity (Fig. 17). Combined image generator and an optical correction system 
106 is disclosed in detail in United States Patents 5,050,966 for an OPTICAL COMBINER 
COLLIMATING APPARATUS; 4,859,031 for an OPTICAL COLLIMATING 
APPARATUS; 4,900,133 for a HEADS-UP DISPLAY COMBINER UTILIZING A 

10 CHOLESTERJC LIQUID CRYSTAL ELEMENT; 4,987,410 for a MULTIPLE IMAGE 
FORMING APPARATUS; and 5,408,346 for an OPTICAL COLLIMATING DEVICE 
EMPLOYING CHOLESTERIC LIQUID CRYSTAL AND NON-TRANSMISSIVE 
REFLECTOR, the disclosures of which are hereby incorporated herein by reference and 
will not be repeated. Suffice it to say, combined image generator and optical correction 

15 system 106 includes a light source 108 which generates broad band white light which is 
gathered and reflected by a parabolic reflector 110. In the illustrative embodiment, light 
source 108 is a tungsten halogen incandescent lamp. The light rays then pass through a 
dielectric green filter 112 which passes light in a specific region of the green portion of the 
spectrum and through a hot mirror 1 14 which removes the infrared content of the 

20 spectrum. Light then passes through a holographic diffuser 116 which homogenizes and 
shapes the light pattern. The light rays then pass through a monochrome liquid crystal 
display with opposing linear polarizers 118 which is supplied with a video signal by image 
processor 18. Items 108-118 make up image generator 74, which, in the illustrative 
embodiment, is a transmissive backlit liquid crystal display. However, image generator 74 

25 could additionally be an emissive display or a reflective display, all of which are well 
known in the art. 

Light rays of the image generated by image generator 74 next pass through an anti- 
reflective coated cover glass 120 which is joined with a left-hand circular polarizer 122 
which is bonded to this cover glass. The opposite surface of circular polarizer 122 is 
30 bonded to a lens 124 having a 50/50 dielectric coating. Such dielectric coating allows light 
rays to be both transmitted through the lens and reflected by the lens. The left-hand 
polarized light X' transmitted through lens 124 contacts a cholesteric liquid crystal layer 
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(CLC) 126 which is left-hand polarized, which is what gives efficient reflection of left-hand 
polarized light X\ as illustrated at X'\ Fifty percent (50%) of light rays X" get 
efficiently reflected by the 50/50 beam splitter on lens 124 as right-hand circular polarized 
light X'". Right-hand polarized light X"' is transmitted by CLC layer 126 and passes 
5 through a right-hand circular polarizer 128 and an anti-reflective coated cover glass 130. 

As can be seen by reference to Fig. 17, the optical configuration of lens 124 in 
combination with the left-hand and right-hand circular polarizers 122, 128 and cholesteric 
liquid crystal layer (CLC) 126, provide image magnification as well as collimate the image 
light in order to produce a very long focal distance image. Advantageously, this structure 
10 allows image portions from multiple image capture devices to be tiled into a unitary image. 
Fig. 17 illustrates an approach using a single image generator. Merging of multiple image 
portions would require additional combined image generator and optical correction 
systems. Although image generators 74 for each of the image portions are laterally spaced 
apart from each other, the amplification produced by combined image generator and optical 
15 correction system 106 causes the image portions to merge at their periphery. Fig. 17 

illustrates an approach using a single image generator. Merging of multiple image portions 
would require additional combined image generators and optical correction systems. Other 
optical elements such as prisms, or other lenses, may be necessary to merge images to form 
a unitary image. Although the invention is illustrated with a combined image generator and 
20 optical correction system using cholesteric liquid crystal optical processing, other optical 
correction systems, as are known in the art, may be used. What is required is that the 
optical system generally collimates the light generated by the image generator and, 
preferably, provides amplification to the generated image. 

In the illustrated embodiment, rear image 42, synthesized from the output of image 
25 capture devices 14, 16, has a lateral width verses vertical height aspect ratio that is between 
approximately 4:1 and 2:1 . Most preferably, the aspect ratio of image 42 is 8:3. This 
allows a panoramic view rearwardly of the vehicle with an optimum information content 
while reducing display of irrelevant information. The aspect ratio of display 20 may be 
different from that of the displayed synthesized image 42. The remaining portion of the 
30 display, either above or below image 42, may be utilized to display images other than 
synthesized image 42. For example, the remaining portion of the display can be used to 
display auxiliary information such as one or more vehicle-operating parameters, such as 
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vehicle speed indicia 56, heading indicia 54, or turn signal indicia 58. Alternatively, the 
remaining portion of the display can be a reconfigurable high-information content display 
area to selectively display various types of information. Such information may include 
incoming facsimile or pager information, phone numbers, and navigational aids including 
5 pull-up maps, route guidance information, global positioning system (GPS) data, intelligent 
vehicle highway system (IVHS) information, as well as radio and environmental system 
control settings, and the like. Display 20 is especially useful for displaying such 
alternative data. Because display 20 has a very long focal length, the driver may consult 
the alternative data by switching the gaze of the driver between forward field of view 70 

10 and to display 20 which does not require extensive refocusing of the driver's eyes. This 
allows the driver to consult the alternative data quickly with reduced fatigue and 
distraction. The content of the auxiliary information displayed may be user-selectable by a 
keypad, trackball, or other input device on the dashboard, steering column, or other 
position readily accessible to the driver. 

15 Although various camera devices may be utilized for image capture devices 14, 16, 

an electro-optic, pixilated imaging array, located in the focal plane of an optical system, is 
preferred. Such imaging array allows the number of pixels to be selected to meet the 
requirements of rearview vision system 12. The pixel requirements are related to the 
imaging aspect ratio of the respective image capture devices, which, in turn, are a function 

20 of the ratio of the vertical-to-horizontal field of view of the devices, as i,s well known in the 
art. In the illustrated embodiment, the imaging aspect ratio of side image capture devices 
14 is 2:1 and the image aspect ratio of central image capture device 16 is variable down to 
0.1:1. Such aspect ratio will produce images which will not typically match that of 
commercially available displays. A commercially available display may be used, however, 

25 by leaving a horizontal band of the display for displaying alpha-numeric data, such as 
portions of an instrument cluster, compass display, or the like, as illustrated in Fig. 3. 

In the illustrated embodiment, image capture devices 14, 16 are CMOS imaging 
arrays of the type manufactured by VLSI Vision Ltd. of Edinburgh, Scotland, which are 
described in more detail in co-pending United States patent application Serial No. 

30 08/023,918 filed February 26, 1993, by Kenneth Schofield and Mark Larson for an 

AUTOMATIC REARVIEW MIRROR SYSTEM USING A PHOTOSENSOR ARRAY, 
the disclosure of which is hereby incorporated herein by reference. However, other 
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pixilated focal plane image-array devices, which are sensitive to visible or invisible 
electromagnetic radiation, could be used. The devices could be sensitive to either color or 
monochromatic visible radiation or near or far infrared radiation of the type used in night- 
vision systems. Each image capture device could be a combination of different types of 
5 devices, such as one sensitive to visible radiation combined with one sensitive to infrared 
radiation. Examples of other devices known in the art include charge couple devices and 
the like. 

Preferably, image capture devices 14 and 16 are all mounted at the same vertical 
height on vehicle 10, although compromise may be required in order to accommodate 
10 styling features of the vehicle. The horizontal aim of image capture devices 14 and 16 is 
preferably horizontal. However, the portion of the image displayed is preferably biased 
toward the downward portion of the captured image because significantly less useful 
information is obtained above the horizontal position of the image capture devices. 
Each image-capturing device 14, 16 is controlled by appropriate supporting 
15 electronics (not shown) located in the vicinity of the imaging array such that, when 

operating power is supplied, either an analog or a digital data stream is generated on an 
output signal line supplied to image processor 18. The support electronics may be 
provided partially on the image chip and partially on associated electronic devices. For 
each exposure period, a value indicative of the quantity of light incident on each pixel of 
20 the imaging array during the exposure period is sequentially outputted in a predetermined 
sequence, typically row-by-row. The sequence may conform to video signal standards 
which support a direct view such that, when a scene is viewed by an image-capturing 
device, the image presented on a display represents directly the scene viewed by the image- 
capturing devices. However, when looking forward and observing a displayed image of a 
25 rearward scene, the driver will interpret the image as if it were a reflection of the scene as 
viewed through a mirror. Objects to the left and rearward of the vehicle, as viewed by the 
rearward-looking camera, are presented on the left-hand side of the display and vice versa. 
If this reversal is effected in image processor 18, it may be by the use of a data storage 
device, or buffer, capable of storing all of the pixel values from one exposure period. The 
30 data is read out of the data storage device in a reversed row sequence. Alternatively, the 
imaging array electronics could be constructed to provide the above-described reversal at 
the image-capturing device or at the display. 
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Data transmission between image capture devices 14, 16 and image processor 18 
and/or between image processor 18 and display 20 may be by electrically conductive leads. 
The leads may comprise either a serial or parallel bus. Alternatively, the data transmission 
may be via plastic or glass fiber-optic cable oi an 7\7- link. It is possible, for particulai 
5 applications, to eliminate image processor 18 and direct drive display 20 from image 
capture devices 14, 16 at the pixel level. This may be accomplished by providing an 
interface between the output of image capture device 14, 16 and display 20 which 
synchronously maps image pixels captured by the image capture device, or devices, to the 
display. This synchronous mapping may be accomplished by providing a one-to-one 
10 mapping in which each pixel measurement is communicated to the display. Alternatively, 
the interface may only transmit pixel data which represents changes in the captured image. 
This allows for a reduction in the communication bandwidth necessary to transmit data 
between the image capture device, or devices, and the display. This may be accomplished 
by encoding the pixel data which represents changes in the captured image with additional 
15 data which designates the position of the pixel or other relevant information. 

Communication between the image capture device, or devices, may be multiplexed. 

The data streams from image-capturing devices 14, 16 are combined in image 
processor 18 and directly mapped to the pixel array of display 20. This process is repeated 
preferably at a rate of at least 30 times per second in order to present an essentially real 
20 time video image. The image captured by side image capture device 14 on the right side of 
the vehicle is presented in right image portion 46 and the image from side image capture 
device 14 on the left side of the vehicle is displayed on left image portion 44. The image 
from center image capture device 16 is displayed on central image portion 48. The three 
image portions 44-48 are presented in horizontal alignment and adjacent to each other. 
25 However, the composite image may be positioned at any desired vertical position in the 
display 20. It is also possible to display image portions 44-48 on separate image devices 
which are adjacent each other. 

In vision system 12, side image capture devices 14 are positioned preferably at a 
forward longitudinal position on vehicle 10 and center image capture device 16 is 
30 positioned at a rearward longitudinal position on the vehicle. As best seen by reference to 
Fig. 7, this positioning creates a difference in the vertical angle between each side image 
capture device 14 and center image capture device 16 with respect to a fixed location P, 
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that is a distance D, behind the vehicle. This difference in sensing angle will cause each 
side image capture device 14 to image an object located at P, on a horizontal row of pixels 
that is different from the horizontal row of pixels that center image capture device 16 will 
image the same object. If the image is below the horizontal centerline of the image capture 
5 device, it will be imaged on a lower row of pixels by center image capture device 16 than 
the row of pixels it will be imaged by the side image capture devices 14, as illustrated in 
Fig. 7. This mismatch between horizontal pixel rows of the captured image is furthermore 
a. function of the distance of the captured image from the rear of the vehicle. This can be 
understood by reference to Fig. 11 which presents a chart 90 having a first column 92 of 
10 pixel lines nl , measured from the array centerline, at which an object will be imaged by 
side image capture device 14 and a second column 94 of pixel lines n2, measured from the 
array vertical centerline, at which the same object will be imaged by center image capture 
device 16. The result is that an object, which is captured by both side and center image 
capture devices 14, 16, will be vertically disjointed at the boundary of the displayed image, 
15 if the object is captured by more than one image capture device. The amount of 

disjointment will be greater closer to the vehicle and less at further distances. If the object 
is elongated in the horizontal direction, such as earth 1 s horizon, bridges; or cross-markings 
on highways, then the object will appear to be either broken or crooked. 

In order to provide uniform display of laterally elongated images, a rearview vision 
20 system 12' is provided having a central image portion 48* which is processed differently 
from the image display portions 44' and 46* produced by the side image capture devices 
(Fig. 8). Central image portion 48* is reduced vertically, or compressed, by removing 
specified scan lines, or pixel rows, from the image captured by center image capture device 
16 in a graduated fashion. The difference in the pixel line at which an object will be 
25 imaged by each of the side and center image capture devices is a function of the distance D 
of the object from the rear of the vehicle, with a greater variation occurring at shorter 
distances and the variation reducing to zero for infinite distances. Therefore, the 
compression of central image portion 48* is non-linear, with substantially no compression 
at the vertical center of the image and greater compression at greater distances above and 
30 below the vertical center point of the image. This is accomplished by removing specific 
lines from the center display in a graduated fashion with a greater number of lines removed 
further from the vertical center of the image. The removed lines may be merely discarded 
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in order to vertically reduce the image. Alternatively, the data contained in the removed 
lines may be utilized to modify the value of adjacent pixels above and below the removed 
line in order to enhance the quality of the compressed image. Averaging, median filtering, 
or other such known techniques may also be used. 

5 Each of right image portion 46' and left image portion 44' includes an upper 

portion 64 which extends above the compressed upper portion of the central image portion 
48'. In the illustrated embodiment, upper portions 64 are deleted in order to present a 
uniform upper horizontal boundary for display 20\ In the illustrated embodiment, the 
mismatch between the lower horizontal boundary of central image portion 48 1 and each of 

10 the left and right image portions provides a dead space 66 which provides a visual prompt 
to the user of the approximate location of the rearward corners S of vehicle 10. This dead 
space 66 in the image displayed on display 20' approximates the footprint occupied by 
vehicle 10 when viewed from point C. This is particularly useful because it provides a 
visual indication to the driver that a vehicle passing vehicle 10, as viewed in either left 

15 image portion 44' or right image portion 46', is at least partially adjacent vehicle 10 if the 
image of the approaching vehicle is partially adjacent to dead space 66. 

In ah alternative embodiment, the vertical compression technique may be applied to 
only a lower vertical portion of central image portion 48'. In most driving situations, 
objects imaged by rearward-facing image capture devices above the horizon are at a long 

20 distance from the vehicle while those below the horizon get progressively closer to the 
vehicle in relation to the distance below the horizon in the displayed image. Therefore, 
compression of the upper vertical portion of the central image portion may be eliminated 
without significant reduction in performance. 

Compression of the central image portion may also advantageously be provided 

25 horizontally, as well as vertically. Spatial separation of center image capture device 16 
from side image capture devices 14 causes similar distortion, as that described above, in 
the horizontal direction. This effect is spherical in nature and would require a more 
complex corrective action, such as compressing the image based upon the removal of pixels 
from an approximation to concentric circles centered on the center of the imaging array, or 

30 other techniques which would be apparent to those skilled in the art. 

A rearview vision system 12* * includes an image display 20' ' having a compressed 
central image portion 48" and left and right image portions 44" and 46* 1 , respectively 
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(Fig. 10). A border 50' between left side image 44* 1 and central image 48" includes a 
vertical central border portion 50a \ an upper border portion 50b\ and a lower border 
portion 50c'. Upper border portion 50b 1 and lower border portion 50c' diverge -laterally 
outwardly, vertically away from central portion 50a 1 . A border 52' between central image 
5 portion 48" and right image portion 46' * includes a central boundary portion 52a \ an 

upper boundary portion 52b 1 , and a lower boundary portion 52c 1 . Upper boundary portion 
52b 1 and lower boundary portion 52c 1 diverge laterally outwardly vertically away from 
central portion 52a 1 . This creates an upper portion of central image portion 48* 1 and a 
lower portion of central image portion 48' 1 which extend beyond the center portion thereof. 
10 This configuration is based upon the realization that the surface of the road immediately 

behind the vehicle is captured by central image capture device 16. Likewise, the horizontal 
plane above the vehicle, which is symmetrical with the road surface, is captured by the 
center image capture device. This may be seen by referring to point P in Fig. 10, which 
illustrate the points where the effective radius 68 of the virtual image capture device 
15 intersects dead zones 30 and by referring to point S in Fig. 10 which illustrates the corners 
or the rear of the vehicle (S). 

The image displayed on display 20* ' includes a dead space 66* having diverging 
lateral sides 68a, 68b. Diverging sides 68a and 68b are configured in order to extend in 
the direction of travel of vehicle 10 which is parallel to lane markings of a highway on 
20 which vehicle 10 is travelling. This further enhances the visual perception of the driver by 
providing a visual clue of the location of images appearing on display 20" with respect to 
the vehicle 10. Side portions 68a f 68b, in the illustrated embodiment, are natural 
extensions of lower boundary portions 50c 1 and 52c* and extend from point S on each 
respective side of the vehicle to point R, which represents the intersection of the lower 
25 extent of the vertical field of view 40 of each side image capture device 14 with the 
pavement (Fig. 7). 

Rearview vision systems 12* and 12' ' utilize a displayed synthesized image which 
takes into account the use of perspective in enhancing the driver's understanding of what is 
occurring in the area surrounding the vehicle. The images produced on displays 20' and.. 
30 20' 1 effectively remove the vehicle bodywork and replace the bodywork with a vehicle 
footprint as would be viewed by virtual camera C. The image displayed on display 20*' 
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further includes perspective lines which further enhance the roll of perspective in the 
driver's understanding of what is occurring. 

In order to further enhance the driver's understanding of what is occurring in the 
area surrounding the vehicle, a rearview vision system 12'" includes a display 20" 1 having 
5 image enhancements (Fig. 6). In the illustrative embodiment, such image enhancements 
include graphic overlays 70a, 70b which are hash marks intended to illustrate to the driver 
the anticipated path of movement of vehicle 10. In the illustrated embodiment, the 
anticipated vehicle motion is a function of the vehicle direction of travel as well as the rate 
of turn of the vehicle. The forward or rearward direction of vehicle travel is determined in 

10 response to the operator placing the gear selection device (not shown) .in the reverse gear 
position. The degree of turn of the vehicle may be determined by monitoring the 
movement of the vehicle steering system, monitoring the output of an electronic compass, 
or monitoring the vehicle differential drive system. In the embodiment illustrated in Fig. 
6, the configuration of graphic overlays 70a, 70b indicates that the vehicle is in reverse 

15 gear and that the wheels are turned in a manner that will cause the vehicle to travel toward 
the driver's side of the vehicle. If the wheels were turned in the opposite direction, graphic 
overlays 70a, 70b would curve clockwise toward the right as viewed in Fig.- 6. If the 
vehicle's wheels were straight, graphic overlays 70a, 70b would be substantially straight 
converging lines. If the vehicle is not in reverse gear position, graphic overlays 70a, 70b 

20 are not presented. Other types of graphic overlays of the displayed image are 
comprehended by the invention. 

Horizontal grid markings on the display may be provided to indicate distances 
behind the vehicle at particular markings. Such grid would allow the driver to judge the 
relative position of vehicles behind the equipped vehicle. In one embodiment, short 

25 horizontal lines are superimposed on the displayed image at regular rearward intervals in 
horizontal positions which correspond to the boundaries of the lane in which the vehicle is 
travelling. In order to avoid confusion when the vehicle is travelling in a curved path, 
from a lack of correspondence between the graphic overlay and the road, a signal indicative 
of the vehicle's rate of turn may be taken into account when generating the graphic 

30 overlay. In this manner, the distance indications may be moved laterally, with reduced 
horizontal separation,- to correspond to the positions of the curved lane boundaries and 
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vertically on the image to compensate for the difference between distances along a straight 
and curved path. 

Another image enhancement is to alter the appearance of an object in a particular 
zone surrounding the vehicle in order to provide an indication, such as a warning, to the 
5 driver. As an example, a vehicle that is too close to the equipped vehicle for safe-lane 
change, may be displayed in a particular color, such as red, may flash, or otherwise be 
distinguishable from other images on the display. Preferably, the speed of the equipped 
vehicle 10, which may be obtained from known speed transducers, may be prbvided as an 
input to the rearview vision system in order to cause such warning to be a function of the 
10 vehicle speed which, in turn, affects the safe separation distance of vehicles. The operation 
of the turn signal may also be used to activate such highlighting of other road users or to 
modify the scope of the image displayed. In order to determine the distance of objects 
behind vehicle 10, a separate distance-measuring system may be used. Such separate 
system may include radar, ultrasonic sensing, infrared detection, and other known distance- 
15 measuring systems. Alternatively, stereoscopic distance-sensing capabilities of side image 
capture devices 14 may be utilized to determine the separation distance from trailing 
objects utilizing known techniques. 

Thus, it is seen that the image displayed on display 20-20" 1 may be different under 
different circumstances. Such different circumstances may relate to the vehicle's direction 
20 of travel, speed, rate of turn, separation from adjacent objects, and the like. 

Various other forms of image processing may be utilized with rearview vision 
system 12-12 ,M . Luminant and chrominant blending may be applied to the images captured 
by image capture devices 14, 16 in order to produce equality of the image data whereby the 
image portions appear as if they were produced by one image capture device. The dynamic 
25 range of the image capture devices may be extended in order to provide high quality 
images under all lighting conditions. Furthermore, individual pixel groups may be 
controlled in order to selectively compensate for bright or dark spots. For example, anti- 
blooming techniques may be applied for bright spots. Multiple exposure techniques may 
be applied to highlight dark areas. Image morphing and warping compensation techniques 
30 may additionally be applied. Resolution of the image capture devices and display may be 
selected in order to provide sufficient image quality for the particular application. 
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A heater may be applied to each image capture device in order to remove dew and 
frost that may collect on the optics of the device. Although, in the illustrative embodiment, 
the optical -center-line- of the camera coincides with the field of view, particular applications 
may result in the centerline of the camera pointing in a direction othei than the centeriint 
5 of the field of view. Although, in the illustrative embodiment, the image capture devices 
are fixed, it may be desirable to provide selective adjustability to the image capture devices 
or optical paths in particular applications. This is particularly desirable when the system is 
used on articulated vehicles where automated and coordinated camera aim may be utilized 
to maintain completeness of the synthesized image. 

10 When operating the vehicle in the reverse direction, it may be desirable to provide 

additional data concerning the area surrounding the immediate rear of the vehicle. This 
may be accomplished by utilizing non-symmetrical optics for the center image capture 
device in order to provide a wide angle view at a lower portion of the field of view. 
Alternatively, a wide angle optical system could be utilized with the electronic system 

15 selectively correcting distortion of the captured image. Such system would provide a 
distortion-free image while obtaining more data, particularly in the area surrounding the 
back of the vehicle. 

The invention additionally comprehends the use of more than three image capture 
devices. In addition to side image capture devices positioned at the front sides of the 

20 vehicle and a center image capture device positioned at the center rear of the vehicle, 

additional image capture devices may be useful at the rear corners of the vehicle in order to 
further eliminate blind spots. It may additionally be desirable to provide an additional 
center image capture device at a higher elevation in order to obtain data immediately 
behind the vehicle and thereby fill in the road surface detail immediately behind the 

25 vehicle. Such additional detail is particularly useful when operating the vehicle in the 

reverse direction. Of course, each of the image capture devices could be a combination of 
two or more image capture devices. 

Although the present invention is illustrated as used in a rearview vision system, it 
may find utility in other applications. For example, the invention may be useful for 

30 providing security surveillance in an area where a building or other object obstructs the 
view of the area under surveillance. Additionally, the invention may find application in 
night-vision systems and the like. For example, the invention may be applied to forward- 
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facing night-vision systems, or other vision enhancement systems such as may be used in 
adverse weather or atmospheric conditions such as fog, applied to provide an enhanced 



virtual camera located rearwardly of the driver, taking advantage of the perspective 
5 features of the image. 

A rearview vision system 150 is provided which, in addition to displaying a rear 
image on display 20 which is synthesized by image processor 18 from the output of image 
capture devices 14, 16, also supplies drive signals to an electrically operated optical device 
such as electro-optic mirror 152, an electro-optic window 154, or both. Even though a 
10 panoramic view rearward of the vehicle is displayed on display 20, it may be desired to 
provide the driver with a rearview mirror of the type which has conventionally been 
provided on vehicles. One such mirror is an electro-optic mirror, such as an 
electrochromic mirror, a liquid crystal mirror, or a solenoid-operated prismatic mirror and 
the like. Additionally, vehicles may be provided with electro-optic windows, such as 
15 sunroofs, rear windows, side windows, and the like, which change transmissivity in 
response to a drive signal to a partial light transmittance level. In United States patent 
application Serial No. 08/023,918 filed February 26, 1993, by Kenneth Schofield and Mark 
Larson for an AUTOMATIC REARVIEW MIRROR SYSTEM USING A 
PHOTOSENSOR ARRAY, the disclosure of which is hereby incorporated herein by 
20 reference, a technique is disclosed for producing a drive signal for an electrically operated 
optical device, such as an electro-optic mirror or window, from the image captured by a 
rearward-facing array. Utilizing the techniques disclosed therein, image processor 18 
produces a drive signal on line 156 in order to control the partial reflectance level of 
electro-optic mirror 152 and a drive signal on line 158 in order to control the partial light 
25 transmittance level of electro-optic window 154. 

A rearview vision system 160 is provided which includes a near infrared 
illumination device 162 in order to enhance an image captured by image capture devices 
14, 16 (Fig. 19). In the illustrated embodiment, infrared illumination device 162 
illuminates an area immediately behind the vehicle. Preferably, the output of illumination 
30 device 162 has a greater near infrared light output than visible light output. This allows an 
enhanced image to be captured by the image capture device without increasing the visible 
light perceived by drivers surrounding the vehicle. Infrared illumination device" 162 may 




ircximates a forward-facing view from a single 
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be actuated in response to the vehicle being placed in reverse gear. This provides backup 
illumination which is greatly enhanced without having an increased effect on other drivers. 
Alternatively, infrared illumination devices may be positioned, for example, at other 
locations on the side or even the front of a vehicle in ordei to enhance mc image captured 
5 by the image capture device or devices. This is especially useful in order to utilize 
rearview vision system 160 with a large truck, such as a trailer truck. This infrared 
illumination device may flood the area around the trailer with infrared light in order to 
enhance the image captured by the image capture device, or devices, without distracting 
other drivers. 

10 Image capture device 14, 16 may include a housing 164 in which an antenna 166 is 

positioned. This provides a convenient and functional location for a receiving antenna, 
. such as the type used with a global positioning system, cellular telephone, garage door 
opener, radar distance sensing device, and the like, as disclosed in patent application Serial 
No. 08/569,851 filed by Desmond J. O'Farrell, Roger L. Veldman and Kenneth Schofield 

15 for a VEHICLE GLOBAL POSITIONING SYSTEM, the disclosure of which is hereby 
incorporated herein by reference. A heater 168 may be associated with the image capture 
device in order to stabilize the temperature of the device in low ambient temperature 
conditions. A similar heater may be supplied in display 20 in order to improve its 
performance in low ambient temperature conditions. A heater control 170 is provided in 

20 order to control the energization of heater 168 and, if utilized, the heater in the display. 
Heater control 170, preferably, energizes heater 168 prior to the vehicle being started. 
This allows the temperature of the image capture device to be elevated to a more desirable 
temperature prior to the driver operating the vehicle. This may be accomplished by heater 
control 170 being a proximity detector which detects a device carried by the driver as the 

25 driver approaches the vehicle. Alternatively, heater control 170 may be responsive to a 
signal produced by a remote keyless entry device concurrently with the doors being 
activated. Alternatively, heater control 170 may be responsive to the vehicle device being 
opened. 

A rearview vision system 172 is provided which provides an output, generally 
30 referred to at 174, from image processor 18 to display 20. Output 174 provides an 

indication when an object bears a predetermined relationship to the vehicle. Such object 
may be of interest because the object is in a blind spot of the vehicle, may be tailgating the 
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vehicle, or may be an object in front of the vehicle which the vehicle is too close to. 
Display 20 may respond to output 174 by highlighting the displayed vehicle, such as by 
displaying the vehicle in an artificial color, such as-red, by flashing the image- of-the 
venire, or, otherwise, drawing the attention ok ihe driver to the vehicle. Output 174 may 

5 be developed by image processor 18 from the outputs of image capture devices 14, 16. 
This may be accomplished by, for example, utilizing redundant image portions captured by 
the image capture devices, even though not displayed by display 20, in order to calculate 
relative position of the object with respect to the vehicle. Alternatively, an object sensor 
176 may be provided in order to supply an output 178 indicative of a predetermined 

10 positional relationship of an object sensed by the object sensor with respect to the vehicle. 
In the illustrated embodiment, object sensor 176 may be a passive infrared sensor which 
senses the presence of an object in the vehicle's blind spot. Alternatively, object sensor 
176 may be a distance-measuring device, such as an active infrared sensor, an ultrasonic 
sensor, a radar sensor, or the like. Such object sensor is especially useful in determining 

15 the separation distance between the vehicle and objects in front of the vehicle. Preferably, 
object sensor 176 has a sensing field of view that is substantially coincident with the field 

i 

of view of one or more of the image capture devices 14, 16. 

A rearview vision system 178 is provided which has the capability of displaying 
stereoscopic images rearward of the vehicle. Rearview vision system 178 includes at least 
20 one pair of image capture devices 14a, which are closely positioned on the vehicle and 
have overlapping fields of view. Because the image capture device pairs are closely 
positioned, they capture substantially the same image but from a slightly different angle. 
This allows image processor 18 to produce a video signal 100' with stereoscopic 
information. This signal is utilized by a stereoscopic display 320 in order to produce a 
25 stereoscopic image rearward of the vehicle. Such stereoscopic displays are known in the 
art. Although one pair of image capture devices are illustrated in Fig. 22, rearview vision 
system 178 may include multiple pairs of image capture devices. This allows a rear image 
to be synthesized from the multiple pairs of image capture devices in order produce a 
panoramic view rearward of the vehicle and stereoscopic image. Preferably, utilizing other 
30 aspects of the invention, the stereoscopic image is a rearward-facing view from a single 
location. 
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A. rearview vision system 180 is provided which produces an indication 182 of road 
line markings. Indication 182 may also indicate road edges. Image processor 18 detects 
the road line markings and the road edges from the images captured by image capture 
devices 14, 16. This feature may be further enhanced by combining it with an infrared 
5 illumination device 162 in order to further illuminate areas behind and around the vehicle 
in order to enhance the image of the road line markings and the road edges. Indication 182 
may be utilized by display 20 in order to provide an indication of the vehicle with respect 
to the road line markings and road edges. The indication may further be utilized by an 
indicator which indicates the relative position of the vehicle in its lane. Additionally, the 
10 indication may be utilized to determine erotic vehicle operation, such as may occur when 
the driver begins to fall asleep, in order to provide a suitable alarm, or the like. 

A rearview vision system 184 is provided with capabilities for infrared 
communication with other vehicles and stationary beacons. Rearview vision system 184 
produces a communication data output 186 which includes communication data decoded 
15 from infrared signals detected by image capture device or devices 14, 16. For example, 
suitable standards may be developed wherein vehicles are equipped with a pair of spaced 
apart infrared transmitters on a forward portion thereof. Image processor 18 may respond 
to the temporal and spatial patterns of infrared signals detected by image capture devices 
14, 16 in order to determine the speed and distance and, thereby, the separation of the 
20 vehicles as well as the rate of change of separation of the vehicles. Such information may 
be communicated to the trailing vehicle by an infrared transmitter (not shown) in order to 
control the speed of the trailing vehicle. This feature provides adaptive cruise control in 
which the speed of the trailing vehicle is controlled according to separation distance with 
leading vehicles. This allows high-speed convoying between vehicles. The communication 
25 system may additionally provide for the identification of emergency vehicles, and the like, 
which may transmit a unique temporal and/or spatial pattern of an infrared signal. The IR 
communication signal may additionally be utilized to receive signals from stationary 
devices, such as location beacons and Intelligent Vehicle Highway System (IVHS) data. 
Because rearview vision system 184 has a field of view which extends generally rearwardly 
30 of the vehicle, the system provides the capability for sensing information after the vehicle 
has passed the beacon. This provides an adjunct to infrared communication systems having 
a field of view generally forward of, or to the side of, the vehicle. 
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A rearview vision system 188 is provided having extended dynamic range (Fig. 25). 
Rearview vision system 188 includes a pair of image capture devices 14 and/or 16, each of 
which has an image-sensing array 190. Each image capture device is capable of operating 
in either a color mode, in which a color image is displayed on display 20, or a 
5 monochrome mode, in which a monochrome image is displayed on display 20. System 188 
includes an image luminance sensor 192 which senses the luminance level of images 
captured by image capture devices 14, 16. Image luminance sensor 192 may switch the 
image capture devices between the color mode and the monochrome mode such that, when 
the image luminance is sufficiently high, the image capture device, or devices, operate in a 
10 color mode. During low image luminance conditions, the image capture device, or 
devices, are operated in a monochromatic mode which does not require as much image 
luminance. This extends the dynamic range of the system. Rearview vision system 188 
may additionally include an exposure control 194 which determines the exposure period for 
capturing each frame by arrays 190. In order to extend the dynamic range of system 188, 
15 exposure control 194 may produce exposure intervals for arrays 190 which vary in length 
from interval-to-interval. Thus, a series of normal exposure intervals may be occasionally 
supplanted by a longer exposure interval during which greater detail of the image may be 
captured. This enhanced image may then be combined with the image captured during the 
shorter intervals into a merged image of enhanced detail. 
20 Rearview vision system 188 may additionally include a plurality of infrared shutters 

196 which are in the optical paths 198 of arrays 190. Each infrared shutter 196 has at least 
one state in which infrared energy is generally not attenuated to array 190. In another 
state, the infrared shutter generally blocks infrared radiation from the array. The state of 
infrared shutters 196 is controlled by image luminance sensor 192. During periods of high 
25 image luminance, sensor 192 may switch the infrared shutters 196 to a state blocking near 
infrared radiation from arrays 190. However, during low image luminance conditions, 
sensor 198 may switch the infrared shutters 196 to a state in which the near infrared energy 
is transmitted to arrays 190. The addition of the near infrared radiation at low luminance 
levels enhances the image luminance sensed by arrays 190. In the illustrated embodiment, 
30 infrared shutters 196 are either electrochromic shutters or liquid crystal shutters, both of 
which are known in the art. Rearview vision system 188 additionally includes means for 
coordinating the image intensity received from multiple image capture devices and 
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displayed on display 20. This allows a balance composite image to be displayed on the 
display. This may be provided by a display intensity control 200 which regulates the 
intensity of the output of both arrays 190 in order to produce color and luminance 
balancing between the multiple image capture devices. Instead of a separate display 

5 intensity control, a direct communication channel may be developed between the image 
capture devices in order to provide color and luminance balancing. 

Each image pixel captured by image capture devices 14, 16 and displayed on 
display 20 has a resolution which affects the amount of image detail displayed. While it is 
desirable to have a high degree of detail of displayed image, the increase in image 

10 resolution produces a commensurate increase in system cost. While it is desirable to 
reduce system cost, this should not be achieved by sacrificing necessary detail in the 
displayed image. According to the invention, it has been determined that sufficient image 
detail may be obtained at a suitable system cost by having a pixel resolution in the range of 
between approximately 2 arc minutes and approximately 8 arc minutes. Preferably, system 

15 resolution is approximately 6 arc minutes. 

It is known in the art to provide imaging array capture devices having mosaic filters 
which mask image radiation in order to produce pixels which respond respectively to red, 
green, and blue light. Because such known pixel filter masks do not adequately absorb 
near infrared radiation, it is known to supply infrared filters in order to block infrared 

20 radiation from the pixels so that the pixels respond to only the designed radiation band. 
However, such additional filters have undesirable characteristics including costs. It has 
been discovered that a pixel filter mask may be made responsive to red, green, or blue light 
while filtering out near infrared by adding appropriate dyes to the dyes making up the filter 
mask. 

25 The heater supplied with each image capture device may include a transparent 

conductive coating applied to a window covering the device lens. Alternative heater 
constructions include. ITO or a series of fine wire mesh. This provides protection of the 
lens of the image capture device from physical harm while allowing moisture and frost to 
be removed from the window. 

30 Various manipulation techniques may be applied to image capture devices 14, 16. 

For example, when the invention is applied to semi-tractor/trailer combinations, the image 
capture devices may be motorized and responsive to signals representing the relative 
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position of the trailer and the cab in order to produce full panoramic view during various 
maneuvers of the vehicle. For example, as the vehicle. turns, which may cause the trailer 
to otherwise block the image captured by a side image capture device, the image capture 
device may pivot to a different panoramic view which is not blocked by the trailer. 
5 Additionally, the panoramic view sensed by the image capture device may be different 
when the trailer is attached than when the trailer is not attached. In a similar fashion, 
rearward-mounted image capture device 16 may be motorized to move vertically upwardly 
and downwardly between a first position when the vehicle is moving in a forward direction 
and a second position when the vehicle is in reverse gear. In the second position, the 
10 motorized camera is directed more downwardly in order to capture images closer to the 
rear of the vehicle which may be contacted by the vehicle. Image capture devices 14, 16 
may be supplied with electrical opto-mechanical zoom devices as well as vibration isolation 
and compensation stabilizing devices. 

Image processor 18 may provide an input to a control for the intensity of the brake 
15 lights, turn signals, and the like for the vehicle. In this manner, the image processor may 
control the intensity of such lights based upon ambient lighting conditions. This allows the 
intensity of the rearward directed lights to be less distracting for following vehicles. 

Additionally, the present invention may be utilized for providing guidance for a 
tractor/trailer or like vehicle backing to a loading dock. Additionally, the invention may 
20 provide guidance for proper alignment of the trailer and tractor which are being joined by 
backing of the tractor. The present invention may additionally provide accident monitoring 
by capturing a predetermined continuous stream of images, such as, for example, 15 
seconds. This running store of image may be frozen upon impact of the vehicle, in order 
to record events leading up to the impact, or may be stopped by a manual input. 
25 Furthermore, the invention may be utilized to alert the driver to an impending rear-end 

collision. The vehicle may respond to such indication by deploying an appropriate device, 
such as a smart headrest or the like. 

Thus, it is seen that the present invention enhances the relationship between the 
driver's primary view and the image presented on the rearview vision system. This is 
30 accomplished in a manner which provides ease of interpretation while avoiding confusion 
so that the driver does not have to concentrate or look closely at the image. In this 
manner, information presented on the display is naturally assimilated. This is 
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accomplished while reducing blind spots so that other vehicles or objects of interest to the 
driver will likely be displayed to the driver. Additionally, the use of perspective allows 
distances to be more accurately determined. 

Changes and modifications in the specifically described embodiments can be carried 
5 out without departing from the principles of the invention, which is intended to be limited 
only by the scope of the appended claims, as interpreted according to the principles of 
patent law including the doctrine of equivalents. 
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The embodiments of the invention in which an exclusive property or privilege is 
claimed are defined as follows: 

1. A rearview vision system for a vehicle, comprising: 

at least two image capture devices positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of said vehicle; and 

a display system which displays an image synthesized from outputs of said image 
capture devices and which approximates a rearward-facing view from a single location. 

2. The rearview vision system in claim 1 wherein said single location is forward of the 
driver with respect to said direction of travel and said rearward-facing view is unobstructed 
by the vehicle. 

3. The rearview vision system in claim 1 wherein said image capture devices are 
spatially separated. 

4. The rearview vision system in claim 1 wherein the displayed image includes a dead 
space which would be occupied by said vehicle in said view. 

5. The rearview vision system in claim 4 wherein said dead space is the size of a 
footprint of said vehicle in said view. 

6. The rearview vision system in claim 4 wherein said dead space includes perspective 
lines which are aligned with said direction of travel. 

7. The rearview vision system of claim 1 wherein said at least two image capture 
devices comprises at least three image capture devices. 

8. The rearview vision system in claim 7 wherein said at least three image capture 
devices include at least two side image capture devices positioned on opposite lateral sides 
of said vehicle and at least one center image capture device laterally between said side 
image capture devices. 

-35- 



WO 96/38319 



PCIYUS96/07382 



9. The rearview vision system in claim 7 wherein said at least three image capture 
devices are at substantially the same height. 

10. The rearview vision system in claim 7 wherein said at least three image capture 
devices are aimed along non-parallel axes. 

1 1 . The rearview vision system in claim 8 wherein the displayed image includes an 
image portion from each of said image capture devices and wherein said image portion 
from said center image capture device has a vertically central portion and vertically upper 
and lower portions, wherein said upper and lower portions are laterally wider than the said 

5 central portion. 

12. The rearview vision system in claim 8 wherein the displayed image includes an 
image portion from each of said image capture devices and wherein said image portion 
from said center image capture device is compressed. 

13. The rearview vision system in claim 8 wherein the displayed image includes an 
image portion from each of said image capture devices and wherein each of said image 
portions has a horizontal field of view that is less than 70 degrees. 

14. The rearview vision system in claim 13 wherein said horizontal field of view of said 
center image portion from said center capture device is less than approximately 12 degrees. 

15. The rearview vision system in claim 14 wherein said horizontal field of view of said 
image portion from said center image capture device is between approximately 3 degrees 
and approximately 8 degrees. 

16. The rearview vision system in claim 1 wherein said display system includes multiple 
contiguous display surfaces, one of said surfaces for each of said images. 

17. The rearview vision system in claim 1 wherein said display system includes a 
display surface for displaying all of said images. 
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18. The rearview vision system in claim 1 wherein each of said image capture devices is 
a CMOS imaging array. 

19. The rearview vision system in claim 8 wherein the displayed image includes an 
image portion from said center image capture device which has a horizontal width that is 
narrower than the horizontal width of image portions from said side image capture devices. 

20. The rearview vision system in claim 19 wherein said horizontal width of said image 
portion from said center image capture device is dynamically adjustable. 

21 . The rearview vision system in claim 20 wherein said horizontal width of said image 
portion from said center image capture device is adjustable in response to vehicle speed. 

22. A rearview vision system for a vehicle, comprising: 

at least three image capture devices mounted to the vehicle and directed generally 
rearwardly with respect to the direction of travel of said vehicle, at least two of said image 
capture devices being side image capture devices respectively mounted on opposite lateral 
5 sides of said vehicle and at least one of said image capture devices being a center image 
capture device mounted between said side image capture devices; and 

a display system which displays an image synthesized from outputs of said image 
capture devices, the displayed image including an image portion from each of said image 
capture devices; 

10 wherein the image portion from said center image capture devices is compressed. 

23. The rearview vision system in claim 22 wherein said image portion from said center 
image capture device is vertically compressed. 

24. The rearview vision system in claim 23 wherein said image portion from said center 
image capture device is compressed by deleting particular ones of said lines. 
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25. The rearview vision system in claim 24 wherein a greater number of lines are 
removed further away from a vertical center of said image portion from said center image 
capture device. 

26. The rearview vision system in claim 24 including adjusting values of adjacent pixels 
as a function of pixel values of removed lines. 

27. The rearview vision system in claim 22 wherein said image portion from said center 
image capture device has a vertically central portion and vertically upper and lower 
portions, wherein said upper and lower portions are laterally wider than said central 
portion. 

28. The rearview vision system in claim 22 including a dead space vertically below said 
image portion from said center image capture device. 

29. The rearview vision system in claim 28 wherein said dead space is the size of a 
footprint of said vehicle in said view. 

30. The rearview vision system in claim 28 wherein said dead space includes 
perspective lines which are aligned with said direction of travel. 

31 . The rearview vision system in claim 22 wherein said display system includes 
multiple contiguous display surfaces, one of said surfaces for each of said image portions. 

32. The rearview vision system in claim 22 wherein said display system includes a 
display surface for displaying all of said image portions. 

33. The rearview vision system in claim 22 wherein each of said image capture devices 
is a CMOS imaging array. 

34. The rearview vision system in claim 22 wherein said at least three image capture 
devices are at substantially the same height on the vehicle. 
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35. The rearview vision system in claim 22 wherein said at least three image capture 
devices are aimed along non-parallel axes. 

36. A rearview vision system for a vehicle having a gear actuator, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of said vehicle; 

a display system which displays an image synthesized from output of said at least 
one image capture device; and 

image enhancement means for enhancing the displayed image including a graphic 
overlay superimposed on said image. 

37. The rearview vision system in claim 36 wherein said graphic overlay includes 
indicia of the anticipated path of travel of the vehicle. 

38. The rearview vision system in claim 37 wherein said graphic overlay is disabled 
when the vehicle's gear actuator is not in reverse gear. 

39. The rearview vision system in claim 37 wherein said indicia has a form that 
responds to the rate of turn of the vehicle. 

40. The rearview vision system in claim 39 wherein said indicia form responds to at 
least one of the vehicle's steering system, the vehicle's differential system, and a compass. 

41 . The rearview vision system in claim 36 wherein said graphic overlay has a form 
that is a function of at least one of the direction or travel and speed of the vehicle. 

42. The rearview vision system in claim 36 wherein said graphic overlay is a distance 
indicia indicating distances behind the vehicle of objects juxtaposed within the grid. 

43. The rearview vision system in claim 42 wherein said indicia has a form that 
responds to the rate of turn of the vehicle. 
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44. The rearview vision system in claim 43 wherein said indicia form responds to at 
least one of the vehicle's steering system, the vehicle's differential system, and a compass. 

45. The rearview vision system in claim 36 wherein said at least one image capture 
device includes a plurality of image capture devices and wherein the displayed image is 
synthesized from outputs of said image capture devices and approximates a rearward-facing 
view from a single location. 

46. The rearview vision system in claim 45 wherein said single location is forward of 
the driver with respect to said direction of travel. 

47. The rearview vision system in claim 36 wherein the displayed image includes a dead 
space which would be occupied by said vehicle in said view. 

48. The rearview vision system in claim 47 wherein said dead space is the size of a 
footprint of said vehicle in said view. 

49. The rearview vision system in claim 47 wherein said dead space includes 
perspective lines which are aligned with said direction of travel. 

50. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; and 

a display system displaying a rear image synthesized from output of said at least one 
image capture device substantially contiguous with the forward field of view of the vehicle 
driver at a focal length that is forward of the vehicle passenger compartment. 

51 . The rearview vision system in claim 50 wherein said display system displays an 
image at a focal length that is within the depth of field of a vehicle driver viewing a distant 
object. 
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52. The rearview vision system in claim 50 wherein said display system displays an 
image either generally above or generally below the central forward field of view of the 
vehicie driver. 

53. The rearview vision system in claim 50 wherein said display system includes an 
opaque viewing portion which blocks view of near field objects located between the 
forward field of view of the vehicle driver and the displayed image. 

54. The rearview vision system in claim 50 wherein said display system defines an 
observation cone within which an image displayed by the display system can be observed 
and further including accommodating means for accommodating variation in relationship 
between a driver's head and said observation cone. 

55. The rearview vision system in claim 54 wherein said accommodating means 
includes adjustment means for adjusting said observation cone in at least one of a 
horizontal and a vertical direction. 

56. The rearview vision system in claim 54 wherein said accommodating means 
includes establishing the rearward field of view captured by said capture device and 
displayed on said display system beyond that normally observed by a driver. 

57. The rearview vision system in claim 50 wherein said display system defines an 
observation cone within which an image displayed by the display system can be observed, 
wherein said observation cone is adjustable in at least one of a horizontal and a vertical 
direction. 

58. The rearview vision system in claim 57 including an adjustment mechanism which 
adjusts said observation cone in said at least one of a horizontal and a vertical direction. 

59. The rearview vision system in claim 58 wherein said adjustment mechanism 
includes an electrically operated actuator which is controllable by a driver-operable switch. 
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60. The rearview vision system in claim 50 wherein said display system has a variable 
luminance output which is varied as a function of ambient light levels. 

61 . The rearview vision system in claim 60 wherein the luminance output of said 
display system is responsive to one of a vehicle headlight activation control, an ambient 
light sensor, and an indication of ambient light level developed by said at least one image 
capture device. 

62. The rearview vision system in claim 60 wherein said display system includes a user 
comfort level setting to establish a desired luminance output level at a particular ambient 
light level. 

63. The rearview vision system in claim 50 wherein said display system includes an 
image generator and an image magnifier which magnifies an image generated by said image 
generator. 

64. The rearview vision system in claim 50 wherein said display device is an opaque 
display. 

65. The rearview vision system in claim 64 wherein said opaque display is in-line with 
the generally forward field of view of the vehicle driver. 

66. The rearview vision system in claim 64 wherein said opaque display is at an angle 
with the generally forward field of view of the driver and directed toward the vehicle 
driver's generally forward field of view by a reflection device. 

67. The rearview vision system in claim 50 wherein said display device is a view- 
through heads-up display which projects an image on an image combiner which combines 
the projected image with an image forward of the vehicle. 

68. The rearview vision system in claim 67 wherein said image combiner includes a 
surface of a vehicle windshield. 
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69. The rearview vision system in claim 67 wherein said display device combines an 
image of a polygon with the rear image in order to border the rear image and differentiate 
the rear image from an image forward of the vehicle. 

70. The rearview vision system in claim 50 wherein said at least one image capture 
device includes at least two image capture devices and wherein said display system displays 
an image synthesized from outputs of said at least two image capture devices which 
approximates a rearward-facing view from a single location. 

71. The rearview vision system in claim 70 wherein the rear image displayed by said 
display system has an aspect ratio that is between approximately 4:1 and approximately 
2:1. 

72. The rearview vision system in claim 71 wherein the rear image displayed by said 
display system has an aspect ratio that is approximately 8:3. 

73. The rearview vision system in claim 72 wherein said display system also displays a 
supplemental image. 

74. The rearview vision system in claim 73 wherein said display system is 
reconfigurable in order to selectively display different supplemental images. 

75. The rearview vision system in claim 73 wherein said supplemental image includes at 
least one of pager information, telephone number listing, global positioning system output, 
map, route guidance information, intelligent vehicle highway system information, vehicle 
radio control settings, vehicle environmental system settings, vehicle speed, vehicle 
heading, and turn signal indicators. 

76. The rearview vision system in claim 50 wherein said display system also displays a 
supplemental image. 
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77. The rearview vision system in claim 76 wherein said display system is 
reconfigurable in order to selectively display different supplemental images. 

78. The rearview vision system in claim 76 wherein said supplemental image includes at 
least one of pager information, telephone number listing, global positioning system output, 
map, route guidance information, intelligent vehicle highway system information, vehicle 
radio control settings, vehicle environmental system settings, vehicle speed, vehicle 
heading, and turn signal indicators. 

79. The rearview vision system in claim 50 wherein the rear image displayed by said 
display system has an aspect ratio that is between approximately 4:1 and 2:1. 

80. The rearview vision system in claim 79 wherein the rear image displayed by said 
display system has an- aspect ratio that is approximately 8:3. 

81- The rearview vision system in claim 53 wherein said housing includes a bezel 
around at least a portion of the displayed image, wherein said bezel has a narrow portion 
adjacent the forward field of view of the vehicle driver. 

82. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; and 

a display system displaying a rear image captured by said at least one image capture 
device, wherein said rear image displayed by said display system is a unitary image having 
an aspect ratio that is between approximately 4:1 and approximately 2:1. 

83. The rearview vision system in claim 82 wherein the rear image displayed by said 
display system has an aspect ratio that is approximately 8:3. 

84. The rearview vision system in claim 82 wherein said display system also displays a 
supplemental image. 
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85. The rearview vision system in claim 84 wherein display system is reconfigurable in 
order to selectively display different supplemental images. 

86. The rearview vision system in claim 84 wherein said supplemental image includes at 
least one of pager information, telephone number listing, global positioning system output, 
map, route guidance information, intelligent vehicle highway system information, vehicle 
radio control settings, vehicle environmental system settings, vehicle speed, vehicle 
heading, and turn signal indicators. 

87. The rearview vision system in claim 82 wherein said at least one image capture 
device includes a plurality of image capture devices and wherein the rear image is 
synthesized from a plurality of images captured by said plurality of image capture devices 
which approximates a rearward-facing view from a single location. 

88. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of said vehicle; and 

a display system which displays an image synthesized from output of said at least 
one image capture device as a unitary image; 

wherein said display system includes at least one image generator associated with 
said at least one image capture device and an optical correction system which amplifies 
image generated by said image generator to increase the focal lengths of the image 
generated by the image generator. 

89. The rear vision system in claim 88 wherein said display device is a view-through 
heads-up display which projects an image on an image combiner which combines the 
projected image with an image forward of the vehicle. 

90. The rearview vision system in claim 89 wherein said image combiner includes a 
surface of a vehicle windshield. 
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91. The rearview vision system in claim 89 wherein said display device combines an 
image of a polygon with the rear image in order to border the rear image and differentiate 
the rear image from an image forward of the vehicle. 

92. The rearview vision system in claim 83 wherein said display system is an opaque 
display. 

93. The rearview vision system in claim 92 wherein said opaque display is in-line with 
the generally forward field of view of the vehicle driver. 

94. The rearview vision system in claim 92 wherein said opaque display is at an angle 
with the generally forward field of view of the driver and directed toward the vehicle 
driver's generally forward field of view by a reflection device. 

95. The rearview vision system in claim 92 wherein said display displays said rear 
image substantially contiguous with the forward field of view of the vehicle driver. 

96. The rearview vision system in claim 95 wherein said display displays said rear 
image laterally substantially directly forward of the vehicle driver. 

97. The rearview vision system in claim 95 wherein said display displays said rear 
image positioned either above or below the central forward field of view of the vehicle 
driver. 

98. The rearview vision system in claim 92 wherein a portion of said display blocks the 
view of near field objects between the vehicle driver's forward field of view and said 
display. 

99. The rearview vision system in claim 88 wherein said display system defines an 
observation cone within which an image displayed by the display system can be observed 
and further including accommodating means for accommodating variation in relationship 
between a driver's head and said observation cone. 



-46- 



WO 96/38319 



PCT/US96/07382 



100. The rearview vision system in claim 99 wherein said accommodating means 
includes adjustment means for adjusting said observation cone in at least one of a 
horizontal and a vertical direction, 

101 . The rearview vision system in claim 99 wherein said accommodating means 
includes said image generator establishing the rearward field of view captured by said 
capture device and displayed on said display system beyond that normally observed by a 
driver. 

102. The rearview vision system in claim 88 wherein said display system defines an 
observation cone within which an image displayed by the display system can be observed, 
wherein said observation cone is adjustable in at least one of a horizontal and a vertical 
direction. 

103. The rearview vision system in claim 102 including an adjustment mechanism which 
adjusts said observation cone in said at least one of a horizontal and a vertical direction. 

104. The rearview vision system in claim 103 wherein said adjustment mechanism 
includes an electrically operated actuator which is controllable by a driver-operable switch. 

105. The rearview vision system in claim 88 wherein said display system has a variable 
luminance output which is varied as a function of ambient light levels. 

106. The rearview vision system in claim 105 wherein the luminance output of said 
display system is responsive to one of a vehicle headlight activation control, an ambient 
light sensor, and an indication of ambient light level developed by said at least one image 
capture device. 

107. The rearview vision system in claim 105 wherein said display system includes a 
user comfort level to establish a desired luminance output level at a particular ambient light 
level. 
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108. The rearview vision system in claim 88 including a plurality of image capture 
devices and a plurality of image generators, each associated with one or more of said image 
capture devices and wherein said optica! correction system merges images captured by said 
image capture device. 



109. A rearview vision system for a vehicle, comprising: 

. at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; and 

an electrically operated optical device which is responsive to a drive signal in order 
to have a partial light transmission level; 

wherein said drive signal is produced from said output of said at least one image 
capture device. 

110. The rearview vision system in claim 109 wherein said electrically operated optical 
device is at least one of an electro-optic rearview mirror, an electro-optic window and a 
prismatic rearview mirror. 

111. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned'on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; and 

a near infrared illumination device positioned to illuminate an area outside of the 
vehicle in order to enhance an image captured by said at least one image capture device. 

112. The rearview vision system in claim 111 wherein said illumination device produces 
substantially greater infrared light output than visible light output. 

113. The rearview vision system in claim 112 wherein said illumination device is 
actuated in response to the vehicle being placed in reverse gear. 
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114. The rearview vision system in claim 112 wherein said illumination device floods 
areas around a trailer of a truck. 

115. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; 

a housing for said image capture device; and 

an antenna positioned in said housing in order to receive electromagnetic radiation 
of a particular characteristic. 

116. The rearview vision system in claim 115 wherein said antenna is one of a global 
positioning system antenna, a cellular telephone antenna, a radar antenna, and a garage 
door opener antenna. 

117. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; and 

designation means for measuring the distance of objects from the vehicle and 
indicating objects which have a predetermined positional relationship to the vehicle. 

118. The rearview vision system in claim 117 wherein said designation means includes a 
distance measuring device. 

119. The rearview vision system in claim 117 wherein said distance measuring device is 
one of a passive infrared sensor, a radar sensor, an active infrared sensor and an ultrasonic 
sensor. 
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120. The rearview vision system in claim 119 wherein said distance measuring device 



has a sensing field of view that is substantially coincident with the field of view of said 
image capture device. 

121. The rearview vision system in claim 117 wherein said designation means produces 
an indication on said rear image of objects which have a predetermined positional 
relationship to the vehicle. 

122. A rearview vision system for a vehicle, comprising: 

at least two image capture devices closely positioned on the vehicle with at least 
partially overlapping fields of view and directed generally rearwardly with respect to the 
direction of travel of the vehicle; and 

a stereoscopic display system displaying a stereoscopic rear image synthesized from 
outputs of said at least two image capture devices. 

123. The rearview vision system for a vehicle wherein said at least two image capture 
devices includes multiple pairs of image capture devices with at least partially overlapping 
fields of view in each pair and wherein said display system displays a rear image 
synthesized from outputs of said multiple pairs of image capture devices which 
approximates a rearward-facing view from a single location. 

124. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; and 

means responsive to said output for detecting road lane markings rearward of the 



125. The rearview vision system in claim 124 wherein said display system displays a 
position of the vehicle with respect to the detected road lane markings.- 



vehicle. 
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126. The rearview vision system in claim 124 including a driver alertness warning which 
is responsive to the position of the vehicle with respect to the detected road lane markings. 

127. The rearview vision system in claim 124 including an infrared light source for 
illuminating a portion of a road rearward of the vehicle. 

128. The rearview vision system in claim 124 including a forward-facing road lane 
marking detector. 

129. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; and 

an infrared communication channel defined from said output which receives 
communications external of the vehicle. 

130. The rearview vision system in claim 129 wherein said infrared communication 
channel is responsive to at least one of a temporal and a spatial pattern of an infrared 
signal. 

131 . The rearview vision system in claim 130 wherein said infrared communication 
channel determines at least one of relative speed and distance between the source of an 
infrared signal and the vehicle. 

132. The rearview vision system in claim 129 wherein said infrared communication 
channel is responsive to infrared signals received from one of a stationary source and a 
source on another vehicle. 

133. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 
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a display system displaying a rear image synthesized from output of said at least one 
image capture device; and 

an interface between said output and said display device which synchronously maps 
image pixels from said output to said display system. 

134. The rearview vision system in claim 133 wherein said interface transmits pixel data 
which represents changes in the captured image. 

135. The rearview vision system in claim 134 wherein said pixel data which represents 
changes in the captured image includes image data and pixel position data. 

136. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; and 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; 

wherein said at least one image capture device includes an extended dynamic range 
to respond to variations in image illumination. 

137. The rearview vision system in claim 136 wherein said image capture device is 
capable of operating in one of a color mode and a monochrome mode and wherein said 
extended dynamic range is achieved by changing between said modes in response to image 
illumination conditions. 

138. The rearview vision system in claim 136 wherein said image capture device 
captures an image during an exposure interval and wherein said extended dynamic range is 
achieved by combining exposure intervals of different lengths. 

139. The rearview vision system in claim 136 wherein said image capture device 
includes an infrared shutter which is selectively operable to generally pass infrared 
radiation during low image illumination conditions and block infrared radiation during high 
image illumination conditions. 
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140. The rearview vision system in claim 139 wherein said infrared shutter is one of an 
electrochromic shutter and a liquid crystal shutter. 

141 . The rearview vision system in claim 136 including a plurality of image capture 
devices and including image intensity coordination between said image capture devices. 

142. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; 

a heater in at least one of said image capture device and said display system; and 

a control which energizes said heater during low temperature conditions prior to 
starting the vehicle. 

143. The rearview vision system in claim 142 wherein said control includes at least one 
of a proximity sensor to sense a user approaching the vehicle and a remote keyless entry 
sensor responsive to a remote keyless entry signal. 

144. The rearview vision system in claim 142 including a heater in both said image 
capture device and said display system. 

145. A rearview vision system for a vehicle, comprising: 

at least one image capture device positioned on the vehicle and directed generally 
rearwardly with respect to the direction of travel of the vehicle; and 

a display system displaying a rear image synthesized from output of said at least one 
image capture device; 

wherein said rear image has a pixel resolution in the range of between 
approximately 2 arc-minutes and approximately 8 arc-minutes. 

146. The rearview vision system in claim 145 wherein said resolution is approximately 6 
arc-minutes. 
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147. The rearview vision system in claim 145 including a plurality of image capture 
devices and wherein said resolution is divided between said image capture devices. 
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(57) Abstract 

A rearview vision system (12) for a vehicle includes at least one image capture device (16) directed rearwardly 
with respect to the direction of travel of the vehicle. A display system (20) displays an image synthesized from output of 
the image capture device. The display system preferably contiguous with the forward field of view of the vehicle driver at a 
focal length that is forward of the vehicle passenger compartment A plurality of image capture devices (14,16) may be 
provided and the display system displays a unitary image synthesized from outputs of the image capture devices which 
approximates a rearward facing view from a single location, such as forward of the vehicle. 
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